Method and system for identifying, tracking, and collecting data on a person of interest

ABSTRACT

A system may be configured to reliably identify, track, and collect data on a person of interest. In some aspects, the system may detect a facial representation within an image, generate a bounding box corresponding to the facial representation, determine an enhanced facial representation based at least in part on the bounding box and an image enhancement pipeline, and extract a plurality of facial features from the enhanced facial representation. Further, the system may determine quality information based on the plurality of facial features, determine that the quality information is greater than a threshold, and store the plurality of facial features in a data structure. In some aspects, the facial features may be used to identify red shoppers and green shoppers in a controlled environment.

BACKGROUND

The present disclosure relates generally to facial recognition, and moreparticularly, to building accurate methods and systems for identifying,tracking, and collecting data on a person of interest.

Some conventional facial recognition systems employ facial landmarks toidentify persons of interests. For instance, some conventional facialrecognition systems identify a person of interest by comparing faciallandmark information to image data potentially including a face of theperson of interest. Typically, facial landmarks are generated from imagedata, e.g., video frames or photographic images. As such, systemaccuracy may be significantly reduced when facial landmark informationis generated from inferior image data. Further, in customer facingsystems, facial recognition inaccuracy may drive economic loss orcustomer dissatisfaction, and squander opportunities for advanced userengagement. For example, in a retail loss prevention context, a facialrecognition system incorrectly identifying a customer as a person ofinterest associated with theft may embarrass the customer andpotentially expose the retailer to legal action.

SUMMARY

The following presents a simplified summary of one or more aspects inorder to provide a basic understanding of such aspects. This summary isnot an extensive overview of all contemplated aspects, and is intendedto neither identify key or critical elements of all aspects nordelineate the scope of any or all aspects. Its sole purpose is topresent some concepts of one or more aspects in a simplified form as aprelude to the more detailed description that is presented later.

The present disclosure provides systems, apparatuses, and methods foraccurately identifying, tracking, and collecting data on a person ofinterest.

In an aspect, a method for generating accurate and reliable facialfeature information comprises detecting a facial representation withinan image, generating a bounding box corresponding to the facialrepresentation, determining an enhanced facial representation based atleast in part on the bounding box and an image enhancement pipeline,extracting a plurality of facial features from the enhanced facialrepresentation, determining quality information based on the pluralityof facial features, determining that the quality information is greaterthan a threshold, and storing the plurality of facial features in a datastructure.

In some implementations, the facial feature information may be employedin a method comprising detecting an alarm event associated with adetection system (e.g., a pedestal of an electronic article surveillancesystem), identifying a face associated with the facial features in asecond image captured by the video capture device, determining that thefacial features are associated with a list of red shoppers, andtriggering an alarm notification based at least in part on detecting thealarm event and determining that the facial features are associated withthe list of red shoppers. In some other implementations, the facialfeature information may be employed in a method comprising detecting ajamming event associated with a pedestal, identifying a face associatedwith the facial features in a second image captured by the video capturedevice, determining that the facial features are associated with a listof red shoppers, and triggering an alarm notification based at least inpart on the detecting the jamming event and the determining that thefacial features are associated with the list of red shoppers.

In some implementations, the facial feature information may be employedin a method comprising identifying a face associated with the facialfeatures in a second image captured by the video capture device,determining that the facial features are associated with a list ofshoppers, collecting customer information associated with the facialfeatures, and sending, based at least in part on the detecting and thedetermining, the customer information to an associate device.

The present disclosure includes a system having devices, components, andmodules corresponding to the steps of the described methods, and acomputer-readable medium (e.g., a non-transitory computer-readablemedium) having instructions executable by a processor to perform thedescribed methods.

To the accomplishment of the foregoing and related ends, the one or moreaspects comprise the features hereinafter fully described andparticularly pointed out in the claims. The following description andthe annexed drawings set forth in detail certain illustrative featuresof the one or more aspects. These features are indicative, however, ofbut a few of the various ways in which the principles of various aspectsmay be employed, and this description is intended to include all suchaspects and their equivalents.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosed aspects will hereinafter be described in conjunction withthe appended drawings, provided to illustrate and not to limit thedisclosed aspects, wherein like designations denote like elements, andin which:

FIG. 1 is a schematic diagram of a system for identifying, tracking, andcollecting data on persons of interest within a controlled environment,according to some implementations.

FIG. 2 is a flow diagram of an example of a method of generatingaccurate and reliable facial landmark information, according to someimplementations.

FIG. 3 is a flow diagram of an example of a method of implementing asystem for accurately identifying, tracking, and collecting data onpersons of interest within a controlled environment, according to someimplementations.

FIG. 4 is block diagram of an example of a computer device configured toimplement a system for accurately identifying, tracking, and collectingdata on persons of interest within a controlled environment, accordingto some implementations.

DETAILED DESCRIPTION

The detailed description set forth below in connection with the appendeddrawings is intended as a description of various configurations and isnot intended to represent the only configurations in which the conceptsdescribed herein may be practiced. The detailed description includesspecific details for the purpose of providing a thorough understandingof various concepts. However, it will be apparent to those skilled inthe art that these concepts may be practiced without these specificdetails. In some instances, well known components may be shown in blockdiagram form in order to avoid obscuring such concepts.

Implementations of the present disclosure provide systems, methods, andapparatuses for accurate identification, tracking, and data collectionon persons of interest within a controlled environment. These systems,methods, and apparatuses will be described in the following detaileddescription and illustrated in the accompanying drawings by variousmodules, blocks, components, circuits, processes, algorithms, amongother examples (collectively referred to as “elements”). These elementsmay be implemented using electronic hardware, computer software, or anycombination thereof. Whether such elements are implemented as hardwareor software depends upon the particular application and designconstraints imposed on the overall system. By way of example, anelement, or any portion of an element, or any combination of elementsmay be implemented as a “processing system” that includes one or moreprocessors. Examples of processors include microprocessors,microcontrollers, graphics processing units (GPUs), central processingunits (CPUs), and other suitable hardware configured to perform thevarious functionality described throughout this disclosure. One or moreprocessors in the processing system may execute software. Software shallbe construed broadly to mean instructions, instruction sets, code, codesegments, program code, programs, subprograms, software components,applications, software applications, software packages, routines,subroutines, objects, executables, threads of execution, procedures,functions, among other examples, whether referred to as software,firmware, middleware, microcode, hardware description language, orotherwise.

In some implementations, one problem solved by the present solution isgenerating reliable facial feature information to employ for accuratelyand efficiently identifying persons of interest despite unfavorableimage data with which to generate the facial feature information. Forexample, this present disclosure describes systems and methods togenerate and employ reliable facial feature information from less thanideal source image data by enhancing the source image data and filteringout sub-standard facial feature information.

Implementations of the present disclosure may be useful for systemdesigners and system operators endeavoring to employ a highly-accuratefacial recognition system. For example, operators of facial recognitionsystems in customer facing contexts have been reluctant to employ facialrecognition for certain measures given the dangers of acting on a falsenegative or false positive. The present solution provides improvedaccuracy in such scenarios by generating and maintaining a data store ofreliable facial feature information.

Referring to FIG. 1, in one non-limiting aspect, a system 100 isconfigured to generate reliable facial feature information, and mayidentify, track, and collect data on persons of interest within acontrolled environment 102 based on the facial feature information. Forexample, system 100 is configured to generate reliable facial featureinformation, and employ the facial feature information to accuratelyidentify and assist preferred shoppers, while detecting unauthorizedactivity by known bad actors, e.g., patrons suspected of stealing.

As illustrated in FIG. 1, the system 100 may identify, track, andcollect data on persons of interest within a controlled environment 102.The system 100 may include a monitoring system 104, one or more videocapture devices 106(1)-(N), a detection system 110, and a communicationnetwork 114. The monitoring system 104 may be configured to monitor thepersons of interest within the controlled environment 102. The one ormore video capture devices 106(1)-(N) may be configured to capture videoframes 108(1)-(N) within the controlled environment 102 and send thevideo frames 108(1)-(N) to the monitoring system 104. Further, thedetection system 110 may be configured to prevent the unauthorizedremoval of articles 112(1)-(N) from the controlled environment 102. Insome aspects, the monitoring system 104, video capture devices106(1)-(N), and detection system 110 may communicate via thecommunication network 114. In some implementations, the communicationnetwork 114 may include one or more of a wired and/or wireless privatenetwork, personal area network, local area network, wide area network,or the Internet.

In some aspects, the detection system 110 may include one or moreantenna pedestals configured to create a surveillance zone at an exit, aprivate area (e.g., fitting room, bathroom, etc.), or a checkout area ofthe controlled environment 102. In some aspects, the detection system110 may transmit exciter signals 116(1)-(N) (e.g., a RFID interrogationsignal) that cause the security tags 118(1)-(N) to produce detectableresponses 120(1)-(N) if an unauthorized attempt is made to remove one ormore articles 112 from the controlled environment 102.

For example, the security tag 118(1) may be attached to the article112(1). Further, the detection system 110 may transmit an exciter signal116(1) that causes the security tag 118(1) to respond with a detectableresponse 120(1). In some aspects, each security tag 118 may beassociated with a tag identifier 122. For instance, the security tag116(1) may be associated with the tag identifier 122(1). Further, thedetectable response 118(1) may include the tag identifier 122(1). Uponreceipt of the detectable responses 120(1)-(N), the detection system 110may send event information 124(1)-(N) to the monitoring system 104and/or trigger an alarm 126. In some examples, the alarm 126 may be avisual notification, audible notification, or electronic communication(e.g., text message, email, etc.).

As illustrated in FIG. 1, the system 100 may include one or moreassociate devices 128(1)-(N) associated with one or more associates130(1)-(N), one or more customer devices 132(1)-(N) associated with oneor more green shoppers 134(1)-(N), and one or more jamming devices136(1)-(N) associated with one or more red shoppers 138(1)-(N). Theassociates 128(1)-(N) may be employed or otherwise associated with anoperator of the controlled environment 102. Further, the associatedevices 130(1)-(N) may be used by the associates 128(1)-(N) infurtherance of their activities within the controlled environment 102.For example, the associates 128(1)-(N) may employ the associate devices130(1)-(N) for customer assistance, customer sales, etc. Some examplesof the associate devices 130(1)-(N) and the customer devices 134(1)-(N)include wearable devices (e.g., optical head-mounted display,smartwatch, etc.), smart phones and/or mobile devices, laptop andnetbook computing devices, tablet computing devices, digital mediadevices and eBook readers, and any other device capable of receivinginformation from the monitoring system 104 or the detection system 110.

The green shoppers 132(1)-(N) may be customers enrolled in a customerloyalty program or having another type pre-existing relationship with anoperator or user of the monitoring system 104 and/or detection system110. The red shoppers 138(1)-(N) may be customers previously-identifiedas having participated in unauthorized activity within the controlledenvironment 102 or another controlled environment. Further, the redshoppers 138(1)-(N) may employ the jamming devices 138(1)-(N) to disruptthe detection system 110 in order to facilitate unauthorized removal ofthe articles 112(1)-(N) from the controlled environment 102 withouttriggering detectable responses 120(1)-(N). For example, a first redshopper 138(1) may employ the jamming device 136(1) to disrupt theoperation of the detection system 110, while a second red shopper 138(2)attempts to exit the controlled environment 102. In some aspects, thejamming devices 138(1)-(N) may be configured to transmit a jammingsignal to frustrate operation of the detection system 110. Further, thedetection system 110 may be configured to detect the presence of ajamming device 136 or an occurrence of a jamming event related totransmission of a jamming signal, and send the event information122(1)-(N) to the monitoring system 104 and/or trigger the alarm 124.Additionally, or alternatively, the detection system 110 may send theevent information 122(1)-(N) to the monitoring system 104, and themonitoring system 104 may detect the presence of a jamming device 136 oran occurrence of a jamming event based upon the event information124(1)-(N). Further, in some aspects, the monitoring system 104 and/orthe detection system 110 may employ machine learning techniques and/orpattern recognition techniques to detect the presence of a jammingdevice 136 or an occurrence of a jamming event.

As illustrated in FIG. 1, the monitoring system 104 may include a facedetector module 140, a boundary generator module 142, an enhancementmodule 144, a feature extractor module 146, a feature evaluator module148, a feature information management module 150, a monitoring module152, feature information 154, and customer information 156. The facedetector module 140 may be configured to detect a face in the videoframes 108(1)-(N) received from the video capture devices 106(1)-(N).For instance, the face detector module 140 may be a facial recognitionsystem employing machine learning and/or pattern recognition techniquesto identify a face within the video frame 106(1). In some aspects, theface detector module 140 may identify a face within the video frame106(1) based at least in part on one or more machine learning modelsconfigured to identify facial landmarks. Further, the boundary generatormodule 142 may be configured to generate boundary representations aroundfaces within video frames 106(1)-(N). For example, the boundarygenerator module 142 may generate a boundary box around a face withinthe video frame 108(1).

The enhancement module 144 may be configured to perform one or moreenhancement processes on the boundary representations to determineenhanced representations. For example, the enhancement module 144 mayperform one or more enhancement methods on the boundary box to generatean enhanced representation. Some examples of enhancement methods includelight correction, shadow effect filtering, and histogram equalization.In particular, the enhancement module 144 may scale a video frame to apre-defined resolution for uniformity and consistency across subsequentenhancement processes, apply a gamma intensity correction (GIC) to theboundary representation to correct lighting variations within theboundary representations, apply a difference of Gaussian filteringalgorithm to the boundary representations to reduce or eliminateshadowing effects within the boundary representations, and/or apply ahistogram equalization to the boundary representations to improve theimage contrast within the boundary representations. In some aspects, theenhancement module 144 may be a graphics processing pipeline thatperforms at least one of the above enhancement methods in any order.

The feature extractor module 146 may be configured to extract facialfeature sets from the enhanced representations. For example, the featureextractor module 146 may be configured to extract a first facial featureset from a first enhanced representation corresponding to the videoframe 108(1). Each facial feature set may include a vector of faciallandmarks. As used herein, in some aspects, a facial landmark may referto a descriptor that may be used to define a face. Some examples of afacial landmark may include the left eyebrow outer corner, left eyebrowinner corner, right eyebrow outer corner, right eyebrow inner corner,left eye outer corner, left eye inner corner, right eye outer corner,right eye inner corner, nose tip, left mouth corner, right mouth corner,eye centers, left temple, right temple, chin tip, cheek contours, lefteyebrow contours, right eyebrow contours, upper eyelid centers, lowereyelid centers, nose saddles, nose peaks, nose contours, mouth contours,the distance between the eye centers, the distance between the nose tipand lips, etc.

The feature evaluator module 148 may be configured to determine aquality of the facial feature sets extracted from the enhancedrepresentations. For instance, the feature evaluator module 148 may beconfigured to determine a quality rating or score of the first facialfeature set. In some aspects, the quality rating or score may be basedat least in part on a face detection confidence score, a facial landmarkscore, a Haar cascade analysis, or an occlusion percentage associatedwith the face representation. For instance, the feature evaluator module148 may determine a face detection score, a facial landmark score,perform Haar cascade analysis, and/or an occlusion percentage for thefirst facial feature set, and generate a quality rating or score basedon the face detection score, the facial landmark score, the results ofthe Haar cascade analysis, and/or the occlusion percentage.

The feature information management module 150 may be configured tomanage storage of the facial feature sets. In some aspects, the featureinformation management module 150 may determine whether to store afacial feature set based upon a measured quality. For instance, thefeature information management module 150 may store the first facialfeature set within the feature information 154 when a quality score orrating associated with the first facial feature set is greater than orequal to a threshold value. In some aspects, the feature informationmanagement module 150 generate a face identifier corresponding to thefirst facial feature set and store vector information including the faceidentifier and the first facial feature set. Further, the faceidentifier may be used by the monitoring system 104 to reference thecorresponding face and/or facial feature set.

Further, in some aspects, the feature information management module 150may replace facial feature sets within the feature information 154 withfacial feature sets of a higher quality. For example, if the featureinformation management module 150 determines that a first facial featureset stored in the feature information 154 and a second facial featureset recently generated by the feature extractor module 146 correspond tosame face, the feature information management module 150 may replace thefirst facial feature set with the second facial feature set within thefeature information 154 when the quality score or rating of the secondfacial feature set is higher than the quality score or rating of thefirst facial feature set.

In some aspects, the feature information management module 150 maycoordinate an enrollment process for the green shoppers 132(1)-(N)and/or the red shoppers 138red shoppers 138(1)-(N) that generatesfeature information 154 and/or customer information 156 for the greenshoppers 132(1)-(N) and/or the red shoppers 138(1)-(N). For instance,the one or more red shoppers 138(1)-(N) may each have a video frame 108or photographic image captured in response to performing one or moreunauthorized activities within the controlled environment 102 or anothercontrolled environment associated with an operator of the controlledenvironment 102. Further, as described in detail herein, the videoframes 108 may be used to generate the facial feature sets for the redshoppers 138(1)-(N). Further, in some aspects, a facial feature sets fora red shopper 138(1) may be distributed to one or more other monitoringsystems (e.g., monitoring systems within a geographic region associatedwith the red shopper 138) as a loss prevention measure. In anotherinstance, the one or more green shoppers 132(1)-(N) may each have avideo frame 108 or photographic image captured in response to forming arelationship with an operator of the controlled environment (e.g.,enrolling a customer loyalty program). Further, as described in detailherein, the video frame 108 may be used to generate the facial featuresets for the green shoppers 132(1)-(N). In addition, the monitoringsystem 104 may collect customer information 156 associated with the oneor more green shoppers 132(1)-(N). In some aspects, the customerinformation 156 may include name, address, email address, demographicattributes, shopping preferences, shopping history, membershipinformation (e.g., a membership privileges), financial information, etc.

Further, the monitoring module 152 may be configured to identify thepresence of the green shoppers 132(1)-(N) and/or the red shoppers138(1)-(N) within the controlled environment 102 based upon the featureinformation 154. For example, the green shopper 132(1) may enter thecontrolled environment 102. In response, the video capture device maycapture a video frame 108(1) including the green shopper 132(1), andsend the video frame 108(1) to the monitoring system 104. Upon receiptof the video frame 108(1), the monitoring system 104 may generate afacial feature set, and the monitoring module 152 may determine that thegenerated facial feature set matches facial feature set stored in thefeature information 154. Further, the monitoring module 152 maydetermine that the facial feature set within the feature information 154corresponds to the green shopper 132(1), and identify customerinformation 156 corresponding to the green shopper 132(1). For example,the monitoring module 152 may identify customer information 156including a picture of the green shopper 132(1), the name of the greenshopper 132(1), the phonetic spelling of the name of the green shopper132(1), recent transaction history of the green shopper 132(1),financial account information, and one or more articles 112 that may beof interest to the green shopper 132(1). In addition, the monitoringmodule 152 may send the customer information 156 to the associate device130(1) so that the associate 128(1) may assist the green shopper 132(1).For example, the associate device 130(1) may generate a notification inresponse to the customer information 156, and present the customerinformation 156 on the graphical displays of the one or more associatedevices 130(1)-(N), e.g., the customer information 156 may be displayedon an optical head-mounted display worn by the associate 128(1) closestto a region of the controlled environment 102 occupied by the greenshopper 132(1). In response, the associate 128(1) may locate the greenshopper 132(1) and perform a formal introduction. Further, the associate128(1) may recommend the one or more articles 112 for purchase, andfacilitate purchase of the articles 112 using the financial information.

In another example, the green shopper 132(1) may exit the controlledenvironment 102. In response, the video capture device may capture avideo frame 108(1) including the green shopper 132(1), and send thevideo frame 108(1) to the monitoring system 104. Upon receipt of thevideo frame 108(1), the monitoring system 104 may generate a facialfeature set, and the monitoring module 152 may determine that thegenerated facial feature set matches facial feature set stored in thefeature information 154. Further, the monitoring module 152 may send anotification to the associate devices 130(1)-(N) that the green shopper132(1) has exited the controlled environment 102.

As another example, the red shopper 138(2) may be within the vicinity ofthe detection system 110(1) of the controlled environment 102. Inresponse, the video capture device 106(1) may capture a video frameincluding the red shopper 138(2), and send the video frame 108(1) to themonitoring system 104. Upon receipt of the video frame 108(1), themonitoring system 104 may generate a facial feature set, and themonitoring module 152 may determine that the generated facial featureset matches a facial feature set stored in the feature information 154.Further, the monitoring module 152 may determine that the facial featureset stored within the feature information 154 corresponds to a redshopper 138. In some instances, the feature information 154 may indicatewhich of the facial feature sets belong to the red shoppers 138(1)-(N)with a red shopper indicator. Alternatively, in some other instances,the feature information 154 may indicate the particular red shopper 138associated with the facial feature set. In addition, the monitoringmodule 152 may receive the event information 122(1) indicating that adetectable response 118(1) has been received from the security tag118(1) at a first period of time. Additionally, the monitoring module152 may trigger the alarm 126 at the detection system 110 and/or one ormore of the associate devices 130(1)-(N) when the video frame 108(1) iscaptured within a threshold period of time of the receipt of thedetectable response 118(1) at the detection system 110(1). Accordingly,the feature information 154 may be used to reinforce the detectionsystem 110, which may suffer from false positives due to reflections,blockages, and other conditions that may cause stray tag reads.

As yet still another example, the first red shopper 138(1) and thesecond red shopper 138(1) may be within the vicinity of the detectionsystem 110(1) of the controlled environment 102. In response, the videocapture device 106(1) may capture video frames 108 including the firstred shopper 138(1) and the second red shopper 138(1), and send the videoframes 108 to the monitoring system. Upon receipt of the video frames108, the monitoring system 104 may generate facial feature sets for thefirst red shopper 138(1) and the second red shopper 138(1),respectively, and the monitoring module 152 may determine that thegenerated facial feature sets match facial feature sets stored in thefeature information 154. Further, the monitoring module may determinethat the facial feature sets stored within the feature information 154correspond to the first red shopper 138(1) and the second red shopper138(1), respectively. In addition, the monitoring module 152 may receivethe event information 122(1) indicating that a jamming event isoccurring at the detection system101(1). Additionally, the monitoringmodule 152 may trigger the alarm 126 at the detection system 110 and/orone or more of the associate devices 130(1)-(N) when the video frame108(1) is captured within a threshold period of time of the detection ofthe jamming event at the detection system 110(1).

FIG. 2 is a flowchart of a method 200 of generating reliable faciallandmark information. The method 200 may be performed by the monitoringsystem 104 or the computing device 400.

At block 202, the video capture device 106(1) may capture a video frame108(1) including a face, e.g., a face of a red shopper 138(1) or a greenshopper 132(1), and provide the video frame 108(1) to the monitoringsystem 104. At block 204, the monitoring system 104 may detect the facewithin the video frame 108(1). At block 206, the monitoring system 104may generate a bounding box around the face within the video frame108(1). At block 208, the monitoring system 104 may perform imageenhancement on the contents of the bounding box. For example, themonitoring system 104 may perform formatting and enhancement processeson the contents of the bounding box. At block 210, the monitoring system104 may extract facial features from the processed video frame. At block212, the monitoring system 104 may perform a quality assessment of theextracted features. For example, the monitoring system 104 may determinea quality score or rating for the extracted features. At block, 214, themonitoring system 104 may determine whether the extracted features meeta threshold level of quality. For example, the monitoring system 104 maydetermine whether the quality score or rating is equal to or greaterthan a threshold quality score or rating.

If the extracted features meet the threshold level of quality, themonitoring system 104 may proceed to block 216. At block 216, themonitoring system 104 may store the extracted features for later use asdescribed in detail herein. If extracted features fail to meet thethreshold level of quality, the monitoring system 104 may proceed toblock 218. At block 218, the monitoring system 104 may discard theextracted features. Further, this process may be repeated on subsequentvideo frames (e.g., one of the video frames 108(2)-(N)) captured by thesame video capture device 106(1) or another video capture device (e.g.,one of the video capture devices 106(2)-(N)).

Referring to FIG. 3, in operation, the monitoring system 104 orcomputing device 400 may perform an example method 300 for implementinga system for accurately identifying, tracking, and collecting data on aperson of interest. The method 300 may be performed by one or morecomponents of the monitoring system 104, the computing device 400, orany device/component described herein according to the techniquesdescribed with reference to FIG. 1.

At block 302, the method 300 includes detecting a facial representationwithin an image. For example, the face detector module 140 may beconfigured to detect a face in a video frame 108(1) received from thevideo capture device 106(1).

At block 304, the method 300 includes generating a bounding boxcorresponding to the facial representation. For example, the boundarygenerator module 142 may be configured to generate a boundary around theface within the video frame 108(1).

At block 306, the method 300 includes determining an enhanced facialrepresentation based at least in part on the bounding box and an imageenhancement pipeline. For example, the enhancement module 144 mayperform one or more enhancement methods on the boundary box to generatean enhanced facial representation. In some examples, the enhancementmethods may form an enhancement pipeline implemented by the enhancementmodule 144. Further, the enhancement pipeline may include formatting thebounding box, applying GIC to the bounded box, applying a difference ofGaussian filtering algorithm to the bounding box, and performinghistogram equalization over the bounding box. Consequently, the method300 may improve the quality of the source image data used to generatefacial feature information, thereby improving the accuracy andreliability of facial feature information when used in a facialrecognition system.

At block 308, the method 300 includes extracting a plurality of facialfeatures from the enhanced facial representation. For example, thefeature extractor module 146 may be configured to the extract a facialfeature set from the enhanced facial representation corresponding to thevideo frame 108(1).

At block 310, the method 300 includes determining quality informationbased on the plurality of facial features. For example, the featureevaluator module 148 may be configured to determine a score or rating ofthe facial feature set extracted from the enhanced facialrepresentation.

At block 312, the method 300 includes determining that qualityinformation is greater than a threshold. For example, the featureinformation management module 150 may determine whether the qualityinformation is greater or equal than a threshold.

At block 314, the method 300 includes storing the plurality of facialfeatures in a data structure. For example, the feature informationmanagement module 150 may store the facial feature set within thefeature information 154 when the quality score or rating associated ofthe facial feature set is greater than or equal to the threshold.Consequently, the method 300 may filter inadequate facial featureinformation, thereby preventing a facial recognition system fromproducing inaccurate results based on poor facial feature information.

Referring to FIG. 4, a computing device 400 may implement all or aportion of the functionality described herein. The computing device 400may be or may include or may be configured to implement thefunctionality of at least a portion of the system 100, or any componenttherein. For example, the computing device 400 may be or may include ormay be configured to implement the functionality of the monitoringsystem 104, video capture device 106, detection system 110, theassociate device 130, the customer device 134, or the jamming device136. The computing device 400 includes a processor 402 which may beconfigured to execute or implement software, hardware, and/or firmwaremodules that perform any functionality described herein. For example,the processor 402 may be configured to execute or implement software,hardware, and/or firmware modules that perform any functionalitydescribed herein with reference to the face detector module 140, theboundary generator module 142, the enhancement module 144, the featureextractor module 146, the feature evaluator module 148, the featureinformation management module 150, the monitoring module 152, or anyother component/system/device described herein.

The processor 402 may be a micro-controller, an application-specificintegrated circuit (ASIC), a digital signal processor (DSP), or afield-programmable gate array (FPGA), and/or may include a single ormultiple set of processors or multi-core processors. Moreover, theprocessor 402 may be implemented as an integrated processing systemand/or a distributed processing system. The computing device 400 mayfurther include a memory 404, such as for storing local versions ofapplications being executed by the processor 402, related instructions,parameters, etc. The memory 404 may include a type of memory usable by acomputer, such as random access memory (RAM), read only memory (ROM),tapes, magnetic discs, optical discs, volatile memory, non-volatilememory, and any combination thereof. Additionally, the processor 402 andthe memory 404 may include and execute an operating system executing onthe processor 402, one or more applications, display drivers, etc.,and/or other components of the computing device 400.

Further, the computing device 400 may include a communications component406 that provides for establishing and maintaining communications withone or more other devices, parties, entities, etc. utilizing hardware,software, and services. The communications component 406 may carrycommunications between components on the computing device 400, as wellas between the computing device 400 and external devices, such asdevices located across a communications network and/or devices seriallyor locally connected to the computing device 400. In an aspect, forexample, the communications component 406 may include one or more buses,and may further include transmit chain components and receive chaincomponents associated with a wireless or wired transmitter and receiver,respectively, operable for interfacing with external devices.

Additionally, the computing device 400 may include a data store 408,which can be any suitable combination of hardware and/or software, thatprovides for mass storage of information, databases, and programs. Forexample, the data store 408 may be or may include a data repository forapplications and/or related parameters not currently being executed byprocessor 402. In addition, the data store 408 may be a data repositoryfor an operating system, application, display driver, etc., executing onthe processor 402, and/or one or more other components of the computingdevice 400.

The computing device 400 may also include a user interface component 410operable to receive inputs from a user of the computing device 400 andfurther operable to generate outputs for presentation to the user (e.g.,via a display interface to a display device). The user interfacecomponent 410 may include one or more input devices, including but notlimited to a keyboard, a number pad, a mouse, a touch-sensitive display,a navigation key, a function key, a microphone, a voice recognitioncomponent, or any other mechanism capable of receiving an input from auser, or any combination thereof. Further, the user interface component410 may include one or more output devices, including but not limited toa display interface, a speaker, a haptic feedback mechanism, a printer,any other mechanism capable of presenting an output to a user, or anycombination thereof.

-   -   Please amend the claims as follows:

1. A method comprising: detecting a facial representation within animage; generating a bounding box corresponding to the facialrepresentation; determining an enhanced facial representation based atleast in part on applying an image enhancement pipeline to contents ofthe bounding box; extracting a plurality of facial features from theenhanced facial representation; determining quality information based onthe plurality of facial features; determining that the qualityinformation is greater than a threshold; and storing the plurality offacial features in a data structure.
 2. The method of claim 1, whereinthe image is a first image, and further comprising: detecting an alarmevent associated with a pedestal; identifying a face associated with theplurality of facial features in a second image; determining that theplurality of facial features are associated with a list of red shoppers;and triggering an alarm notification based at least in part on thedetecting and the determining.
 3. The method of claim 1, wherein theimage is a first image, and further comprising: detecting a jammingevent associated with a pedestal; identifying a face associated with theplurality of facial features in a second image; determining that thefacial features are associated with a list of red shoppers; andtriggering an alarm notification based at least in part on the detectingand the determining.
 4. The method of claim 1, wherein the image is afirst image, and further comprising: identifying a face associated withthe plurality of facial features in a second image; determining that theplurality of facial features are associated with a list of greenshoppers; collecting customer information associated with the pluralityof facial features; and sending, based at least in part on the detectingand the determining, the customer information to an associate device. 5.The method of claim 1, wherein determining the enhanced facialrepresentation comprises at least one of: applying a resolutionformatting method to the bounding box; applying a light correctionmethod to the bounding box; applying a shadow effect filter to thebounding box; or applying a histogram equalization to the bounding box.6. The method of claim 1, wherein determining the quality informationbased on the plurality of facial features comprises at least one:determining a face detection confidence score; determining a faciallandmark score; performing a Haar cascade analysis; or determining anocclusion percentage associated with the facial representation.
 7. Themethod of claim 1, wherein storing the plurality of facial features:assigning a face identifier to the plurality of facial features; andstoring a vector including the face identifier and the plurality offacial features.
 8. The method of claim 1, wherein the facialrepresentation is a first facial representation, the image is a firstimage, the bounding box is a first bounding box, the enhanced facialrepresentation is a first enhanced facial representation, the pluralityof facial features are a first plurality of facial features, and thequality information is first quality information, and furthercomprising: detecting a second facial representation within a secondimage; generating a second bounding box corresponding to the secondfacial representation; determining a second enhanced facialrepresentation based at least in part on the second bounding box and theimage enhancement pipeline; extracting a second plurality of facialfeatures from the second enhanced facial representation; determiningsecond quality information based on the second plurality of facialfeatures; determining that the second quality information is lesser thanthe threshold; and disregarding the second plurality of facial features.9. The method of claim 1, wherein the facial representation is a firstfacial representation, the image is a first image, the bounding box is afirst bounding box, the enhanced facial representation is a firstenhanced facial representation, the plurality of facial features are afirst plurality of facial features, and the quality information is firstquality information, and further comprising: detecting a second facialrepresentation within a second image; generating a second bounding boxcorresponding to the second facial representation; determining a secondenhanced facial representation based at least in part on the secondbounding box and the image enhancement pipeline; extracting a secondplurality of facial features from the second enhanced facialrepresentation; determining that the second plurality of facial featurescorrespond to a same face as the first plurality of facial features;determining second quality information based on the second plurality offacial features; determining that the second quality information isgreater than the first quality information; and replacing, within thedata structure, the first plurality of facial features with the secondplurality of facial features.
 10. A system comprising: management devicecomprising: a memory; and at least one processor coupled to the memoryand configured to: detect a facial representation within an image;generate a bounding box corresponding to the facial representation;determine an enhanced facial representation based at least in part onapplying an image enhancement pipeline to contents of the bounding box;extract a plurality of facial features from the enhanced facialrepresentation; determine quality information based on the plurality offacial features; determine that the quality information is greater thana threshold; and store the plurality of facial features in a datastructure.
 11. The system of claim 10, further comprising a pedestal anda video capture device, and wherein the image is a first image and theat least one processor is configured to: detect an alarm eventassociated with the pedestal; identify a face associated with theplurality of facial features in a second image captured by the videocapture device; determine that the plurality of facial features areassociated with a list of red shoppers; and trigger an alarmnotification based at least in part on the detecting and thedetermining.
 12. The system of claim 10, further comprising a pedestaland a video capture device, and wherein the image is a first image andthe at least one processor is configured to: detect a jamming eventassociated with the pedestal; identify a face associated with theplurality of facial features in a second image captured by the videocapture device; determine that the plurality of facial features areassociated with a list of red shoppers; and trigger an alarmnotification based at least in part on the detecting and thedetermining.
 13. The system of claim 10, further comprising an associatedevice and a video capture device, and wherein the image is a firstimage and the at least one processor is configured to: identify a faceassociated with the plurality of facial features in a second imagecaptured by the video capture device; determine that the plurality offacial features are associated with a list of green shoppers; collectcustomer information associated with the plurality of facial features;and send, based at least in part on the detecting and the determining,the customer information to the associate device.
 14. The system ofclaim 10, wherein to determine the enhanced facial representation the atleast one processor is configured to: apply a resolution formattingmethod to the bounding box; apply a light correction method to thebounding box; apply a shadow effect filter to the bounding box; or applya histogram equalization to the bounding box.
 15. The system of claim10, wherein to determine the quality information based on the pluralityof facial features the at least one processor is configured to:determine a face detection confidence score; determine a facial landmarkscore; perform a Haar cascade analysis; or determine an occlusionpercentage associated with the facial representation.
 16. Anon-transitory computer-readable device having instructions thereonthat, when executed by at least one computing device, causes the atleast one computing device to perform operations comprising: detecting afacial representation within an image; generating a bounding boxcorresponding to the facial representation; determining an enhancedfacial representation based at least in part on applying an imageenhancement pipeline to contents of the bounding box; extracting aplurality of facial features from the enhanced facial representation;determining quality information based on the plurality of facialfeatures; determining that quality information is greater than athreshold; and storing the plurality of facial features in a datastructure.
 17. The non-transitory computer-readable device of claim 16,wherein the image is a first image, and the operations furthercomprising: detecting an alarm event associated with a pedestal;identifying a face associated with the plurality of facial features in asecond image; determining that the plurality of facial features areassociated with a list of red shoppers; and triggering an alarmnotification based at least in part on the detecting and thedetermining.
 18. The non-transitory computer-readable device of claim16, wherein the image is a first image, and the operations furthercomprising: identifying a face associated with the plurality of facialfeatures in a second image; determining that the plurality of facialfeatures are associated with a list of green shoppers; collectingcustomer information associated with the plurality of facial features;and sending, based at least in part on the detecting and thedetermining, the customer information to an associate device.
 19. Thenon-transitory computer-readable device of claim 16, wherein determiningthe enhanced facial representation comprises at least one of: applying aresolution formatting method to the bounding box; applying a lightcorrection method to the bounding box; applying a shadow effect filterto the bounding box; or applying a histogram equalization to thebounding box.
 20. The non-transitory computer-readable device of claim16, wherein determining the quality information based on the pluralityof facial features comprises at least one: determining a face detectionconfidence score; determining a facial landmark score; performing a Haarcascade analysis; or determining an occlusion percentage associated withthe facial representation.